关键词: 词内部模式, 新词语识别, 改进位置成词概率, 首尾单字成词概率

Abstract: As to new word identification problem, this paper proposes a new method for Chinese new word identification based on the inner pattern of word. After repeat finding based on suffix arrays and longest common preffix, it propses the weighting of the improved PWP and inside word probabilities in view of the inner pattern of word. At the meanwhile, the paper uses AV and MI statistics to identify Chinese new words. By comparison, find that this method is effective in recognition of Chinese new words.

Key words: inner pattern of word, new word identification, improved PWP, inside word probabilities